22-4-2020 Questionnaires - Field Trials of Health Interventions - NCBI Bookshelf 


NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health. 


Smith PG, Morrow RH, Ross DA, editors. Field Trials of Health Interventions: A Toolbox. 3rd edition. Oxford (UK): OUP Oxford; 
2015 Jun 1. 


Chapter 14 Questionnaires 


1. Introduction to questionnaires 


The collection of information by asking questions of members of the study population is likely to be a component of 
any health intervention trial. Such information may be relatively simple and straightforward to collect (for example, a 
census of the study population in which the name, age, and sex are recorded for the members of each household in the 
study area) or may be very difficult to elicit reliably from respondents (for example, beliefs about the causes of illness 
or details of income or sexual behaviour). The focus of this chapter is on quantitative surveys, in which data are 
collected by asking the same questions to multiple members of the study population. The responses are recorded in a 
standardized way, either on paper or electronically, and analysed later. Qualitative approaches to investigate the 
beliefs, attitudes, and practices of members of a study population, such as anthropological studies based on participant 
observation, in-depth interviews, or focus group discussions, are discussed in Chapter 15. In this chapter, we discuss 
key issues related to the methods of collection of quantitative survey data. The selection, training, standardization, 
monitoring, supervision, and support of the interviewers to ensure that they do a good job of collecting the data are 
discussed in Chapter 16. 


The commonest approach to the collection of quantitative survey data is through face-to-face interviews where an 
interviewer asks each of the questions and records the participant's answers, either on paper or electronically. The 
major advantages of this method are that the participants do not need to be literate and will usually be familiar with 
this approach. However, it is relatively labour-intensive, since each participant has a questionnaire administered by an 
interviewer on a one-to-one basis. 


In literate populations, questionnaires may be ‘self-administered’, i.e. either a paper questionnaire is distributed to 
study participants that they are asked to complete themselves or the participant is given an electronic device such as a 
computer (desktop, laptop, tablet, PDA) or mobile phone on which they read each question and enter the answer. 
These methods can be ‘audio-assisted’ where the participant can listen to each question being read out and select the 
answer from a list. Such approaches have been successfully used with semi-literate participants where the participants 
can listen to the pre-recorded questions and possible answers and only need to be able to identify and select the 
answer code (such as A, B, or C) (Langhaug et al., 2010). 


The basic principles of planning and designing self-administered questionnaires are similar to those for the 
interviewer-administered questionnaires. Interviews of several respondents at the same time (group interviews or 
focus group discussions) are discussed in Chapter 15, Section 3.3. 


In addition to asking questions, an interviewer may carry out observations. For example, questions about the use of 
bed-nets could be supplemented by inspection, and observations on their location and state of repair. Similarly, the 
participant may be asked to demonstrate how they do something. For example, in a study of diarrhoea, they might be 
asked to show how they would prepare oral rehydration salts or how they wash their hands. 


The methods outlined in this chapter are most appropriate when information on a relatively small number of well- 
defined subject areas is required, for which the responses to enquiries are either numerical (for example, number of 
pregnancies) or may be classified into a small number of different categories (for example, current feeding mode of an 
infant). Even simple items of information may be difficult to elicit accurately, unless adequate research has been 
conducted to find out how questions should be asked and phrased in the study community. The methods described in 
Chapter 15 to obtain such background information are relevant here. 


In a particular trial, the study subjects may be visited and interviewed once only or, more commonly, several times. 
Simple cross-sectional surveys provide an example of the former. An example of the latter would be the collection of 
regular information on child morbidity from the mothers of study children through weekly or fortnightly interviews 
such as might be used for the evaluation of the efficacy of a vaccine against diarrhoeal disease. The first interview 
might be more extensive, with a shorter list of questions asked at each subsequent visit. Intervention trials often 
involve an initial cross-sectional survey, followed by periodic surveys of either the same or different individuals from 
the trial population, the frequency of which will be determined by the nature of the outcome variables under study. 
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In this chapter, the different components of a questionnaire survey are reviewed. The formulation and validation of 
questions to be included are considered in Section 2. Section 3 deals with the construction of the complete 
questionnaire; Section 4 deals with the interviewers, their selection, training, and standardization; Section 5 discusses 
the alternative ways of ‘capturing’ the data, using pen and paper or electronic methods, while Section 6 discusses 
factors relating to the actual interview. 


As with most aspects of field research, there is no satisfactory substitute for experience to know how to formulate and 
administer a questionnaire satisfactorily. The inexperienced investigator would be well advised to seek guidance of 
those who have previously conducted surveys in the study area, if possible, as well as searching for examples of 
questionnaires that have been extensively validated in similar contexts such as national censuses and Demographic 
and Health Surveys (DHS). Those with social science, statistical, and data processing skills are also likely to make 
important contributions. A recently updated guide to questionnaire construction and question design is Woodward and 
Chambers (2012). 


2. The questions 


Quantitative data may be collected in field trials by a series of questions asked of the respondents that are compiled 
into a questionnaire. Additional quantitative data may be obtained by direct observation (for example, of what the 
house's roof is made of or of whether a male has been circumcised), measurement (for example, weight), or after 
taking a tissue sample (for example, haemoglobin level). This section will cover issues related to data that are 
collected through questions. 


2.1. Relation to study objectives, content, and duration 


The questions to be included in a questionnaire should be developed to relate directly to the objectives of the study. 
Usually, at least an outline questionnaire will be drawn up in parallel with the formulation of the protocol for the trial. 
Most grant review committees expect to see such an outline in the trial funding application. 


Questionnaires must be realistic, both in terms of content and length. For example, it may not be possible to obtain 
valid data on highly sensitive questions such as illegal or stigmatized behaviours through a structured questionnaire. 


When a questionnaire survey is being planned, it will often seem attractive to add questions that do not relate directly 
to the objectives of the study but which may be of interest for other reasons (see also Chapter 12, Section 2.1). Asa 
general rule, this temptation should be resisted, as lengthening interview schedules is likely to lead to a higher non- 
participation rate, and time devoted to questions of peripheral interest may be at the expense of time on more 
important questions, with a consequent lowering of the quality of the information collected on the latter. It is good 
practice to go through a draft questionnaire, specifying which objective or important trial outcome each question will 
contribute to, with the aim of deleting any which cannot be clearly justified on these grounds. Nonetheless, in some 
circumstances, it may be desirable to ask other questions if this increases the likelihood of participation in the survey 
or serves to divert attention away from the main questions, in order to reduce the chance of biased responses. For 
example, it may be more acceptable to ask questions about sexual behaviour in the context of a more general 
behavioural survey than to include only questions that concern sexual behaviour. Similarly, if particular adverse 
effects are expected from an intervention, it will usually be best to also include questions about effects thought to be 
unrelated to the intervention, as this may help identify any biases in response between intervention and control groups 
that are not directly attributable to the intervention. 


Few respondents will be willing to complete a questionnaire that takes more than 20—30 minutes, and, even if they do, 
the quality of responses may well decline if the respondent gets bored or tired. In general, it is best if a questionnaire 
can be kept to less than 30 minutes, though this can sometimes be extended if it includes a variety of different 
activities, such as answering questions about photographs or scenarios or taking physical measurements, rather than 
only questions and answers. 


2.2. Development of questions 


A plan for the development of the questions to be included in a questionnaire survey is given in Box 14.1, and Box 
14.2 gives a checklist of points that should be considered in drafting questions. 


Increasingly, standardized questionnaires are being developed and shared. These draw on questionnaires and 
interviews that have been conducted in many countries and studies, and often the questions and responses have been 
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translated into many different languages. An example of this is the Economic and Social Research Council (ESRC) 
question bank (<http://www.surveynet.ac.uk/sqb>), which has hundreds of survey questionnaires in it. With the 
advent of standards for data documentation (see <http://www.ddialliance.org>), searching and browsing for questions 
on particular themes will be easier and more extensive. While it is unlikely that complete questionnaires can be copied 
for new trials, it is important to utilize the resources and knowledge from previous studies to avoid making the same 
mistakes and to build on existing knowledge. 


2.3. Types of question 


Information may be sought on opinions or facts through a questionnaire. The distinction between the two is not 
always clear, but, in general, the collection of data on the latter is easier to plan. Local sensitivities will influence the 
reliability with which either kind of information may be obtained. For example, in some cultures, it is considered 
unlucky to count your children, so asking a parent ‘How many children do you have?’ may be too direct an approach. 


2.3.1. Historical recall 


Information may be sought about the present (for example, ‘Does your child have fever now?") or about the past (‘Did 
your child have an episode of fever in the last month?"). The advantage of asking about the present situation is that 
responses are not susceptible to memory lapses, and furthermore they will usually be more amenable to validation 
(see Section 2.4). The reliability of historical information decreases the further back in time the question relates to, 
and is influenced greatly by the importance of the event to the person (also referred to as its salience). Thus, deaths 
will be remembered better than hospital admissions, which, in turn, will be remembered better than illness episodes 
not requiring hospital admission. To obtain reliable information on mild, or even moderately severe, fevers, diarrhoea, 
or respiratory infections, the recall period probably should not exceed a week. The implication of this for longitudinal 
studies in which these outcomes are of interest, is that at least weekly surveys of the study group will be necessary to 
collect reliable information. 


2.3.2. Open and closed questions 


A ‘closed’ question is one that allows only a defined set of answers which have been anticipated and categorized in 
advance (for example, ‘Do you own a radio?’ 0 = No; 1 = Yes). Replies to an ‘open’ question can take any form and 
should, whenever possible, be recorded in the respondent's own words (for example, *What were the symptoms your 
child had before being taken to the health facility?"). It is possible to ask a question with a closed list of responses in 
an open way, with the answer being assigned to one of a previously compiled list of codes held by the interviewer (for 
example, *What did you eat yesterday?', with a list of types of food on the questionnaire for the interviewer to tick off 
those mentioned). This may produce a different response from asking closed questions about each of the items on the 
list. Reading out the list will remind the respondent of the possibilities but may also tend to produce affirmative 
answers as a gesture to ‘please’ the interviewer or because the respondent is embarrassed to admit that they have not 
eaten a high-status food such as meat. If the information is sufficiently important, both approaches can be used, the 
list of unmentioned possible answers being read out after initial responses are recorded without such prompting. The 
two responses should be recorded separately. For example, against each category, there could be three options: ‘0 = 
No’; ‘1 = Yes, unprompted’; ‘2 = Yes, prompted’. An analogy is medical history taking where questions about 
specific signs and symptoms might be asked after an initial neutral enquiry such as ‘What is the problem today?’. 


In preliminary qualitative investigations, open questions are likely to be preferred to determine the full range of 
possible responses. As a general rule, however, for questions that are to be administered in a large survey, closed 
questions are better, as it is very tedious and time-consuming to go back to code the open answers subsequently. It is 
important that they are the ‘right’ closed questions, of course. This requires careful research and the avoidance of the 
premature administration of a questionnaire that may be simple to administer, code, and analyse but which does not 
provide the information required to meet the study objectives. 


2.4. Validation 


The principles underlying the validation of a questionnaire are similar to those for validating a diagnostic test. The 
objective is to determine to what extent the answers given to a question correspond to the ‘true’ situation. Problems 
arise if there is no independent way of ascertaining what is ‘true’. If a mother is asked ‘Does your child have fever 
now?’, the temperature of a child might be measured independently, and the response to the question validated against 
the direct measurement (by defining temperatures above some limit as fever"). It will usually be impossible to 
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validate the responses to a question such as ‘Did your child have fever yesterday?’. If a ‘gold standard’ exists, i.e. a 
means of obtaining an independent measure of the true response, the sensitivity and specificity of a given question can 
be assessed. The sensitivity of the question 1s the proportion of true positive responses that are reported as positive 
(for example, the proportion of all children with a current fever who are reported as having fever by their mother). 
The specificity is the proportion of true negative responses where the question produces a ‘negative’ response (for 
example, the proportion of all children without a current fever who are classified as not having a fever by questioning 
their mother). The relative importance of sensitivity and specificity in intervention trials is discussed in Chapter 12, 
Section 4.2. 


If there is no ‘gold standard’, other characteristics of the responses to questions must be evaluated to assess their 
usefulness in a particular survey. A minimal requirement for a question should be that the respondent gives the same 
answer to the same question at different times if the circumstances have not changed (i.e. responses should be 
*repeatable"). Also, if different interviewers administer the same question to the same person, the same answer should 
be obtained (i.e. responses should be ‘reproducible’). Repeatability and reproducibility are not a guarantee of validity, 
of course. The question ‘Do you beat your spouse?’ might be answered consistently over time to the same interviewer 
and reproducibly to different interviewers, but it may still be a very poor way of detecting spouse beaters! Also, a man 
might consistently report that his wife is his only sexual partner, even if this is not the case. 


If a question fails to induce consistent answers, either within or between interviewers, it may be because of a fault in 
the question or in the interviewers or be due to the respondent deliberately varying their responses for some reason. 
For example, the respondent might reason that, *if I am being asked the same question a second time, this must be 
because they didn't like my first answer, so I'd better change it^! 


2.5. Translation 


It will often be necessary to translate the questionnaire into local languages. Such translation should be undertaken 
with care and attention to detail, as it is easy for the sense of a question to be changed, sometimes substantially, by the 
translation process. For example, apparent differences in responses to a question asked to those in different language 
groups may be due entirely to variations in the translation processes. Words for some illnesses or concepts may not 
exist in a language, and this may necessitate major changes in the wording of questions. An apparently equivalent 
word may exist, but it may be used in a different way and cover a narrower, or wider, range of conditions. For 
example, there may be several local words used to describe acute respiratory infections, one of which corresponds 
closely to what we mean by pneumonia. Conversely, difficulties may be encountered when one local word is used to 
encompass several different conditions. In studies of meningitis in The Gambia, for example, there was difficulty in 
finding terms to distinguish between a ‘floppy’ and a ‘stiff’ neck (B. Greenwood, personal communication). 


Once a questionnaire has been translated into a local language, it should be independently back-translated into the 
original language. Comparing the original text with the back-translated text will indicate possible areas of confusion 
where attention to the original translation will be required. 


3. The questionnaire 


3.1. Length 


Adequate time must be allowed for the interviewer to solicit the correct responses to all the questions included in a 
questionnaire. The time that an interview will take may be difficult to estimate and may depend on the inherent 
interest of the subject matter to respondents, as well as the amount of time they can spare. The likely duration of an 
interview can be evaluated during pilot testing. Neither the interviewer nor the respondent should feel under time 
pressure to complete the interview. Also, the questionnaire should be long enough to allow the required information to 
be collected, but without unduly inconveniencing the respondent. The work schedule of interviewers should be 
planned, such that they are not tempted to hurry through interviews. In general, it is not a good policy to pay 
interviewers according to the number of interviews completed, unless it is certain that this will not compromise 
quality. Sufficient time must be allocated to allow the interviewer to explain why the survey is being conducted, to 
emphasize the importance of truthful responses, and to reassure the respondent regarding the procedures undertaken 
to ensure the confidentiality of any information divulged in the interview. 


Interviews lasting an hour or more are rarely feasible in the context of a large-scale survey; usually, it is more realistic 
to aim for a maximum of around 30 minutes per interview. Respondents may not complete an interview that is too 
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long, and this may be particularly problematic if crucial questions are towards the end of the questionnaire. Problems 
of compliance may also grow, as the interviewers' reputations go before them. Brevity is especially important if 
repeated follow-up questionnaires are planned. 


Itis a good practice to have the interviewers record the time that each interview starts and finishes. This is one way of 
checking how interviewers spend their days (though it is obviously susceptible to manipulation), and, more 
importantly, it provides a measure, for example, of whether different degrees of attention are being given to those in 
the intervention or control groups, with a consequent possibility of bias. 


3.2. Order of questions 


The initial questions in an interview will seek to verify the identity of the respondent (to ensure the correct person is 
being questioned) and to collect basic demographic information (for example, age, sex, marital status). The most 
sensitive questions should usually be asked in the second half of the questionnaire. This is done to give the 
interviewer time to establish a rapport with the respondent and also so that, if the respondent should be upset by the 
questions and withdraws from the interview, at least this happens after most other information has been collected 
(though such questions should have been weeded out during pilot testing). However, it is usually best not to have the 
most sensitive questions last, to avoid the respondent ending the questionnaire with these at the top of their mind. 
Questions which are not judged to be sensitive should tend to be asked in their order of importance (to the study 
objectives), the most important ones being asked first, to minimize the losses due to any premature cessation of an 
interview. 


Responses to some questions may condition the responses to other questions, and this should be taken into account in 
their ordering. For example, a question asking if the respondent is generally ‘well’, which produces a ‘yes’ response, 
may bias questions about specific illnesses if the respondent feels obliged to justify their overall ‘wellness’. If the 
interest of the study is in specific diseases, it might be better to focus on these first, before questions about general 
health. 


Some questions may seek to obtain the same information in different ways as a validation procedure. If this is done, 
the questions should not be too close together in the questionnaire. 


3.3. Layout 


A questionnaire should be able to be used in the field with, at most, infrequent reference to manuals or instructions. It 
should provide the interviewer with sufficient information to conduct the interview smoothly and without difficulty, 
after suitable training (see Section 4.2). At the same time, it should not be a bulky document, as this may alarm the 
respondent (in terms of the time they think it will take to complete), and it may add to the problem of paper storage 
(see Section 5). Instructions to interviewers may be distinguished from questions to respondents by printing them in a 
different typeface (for example, italics). Each interviewer should be issued with an interviewer's manual (see Section 
4.4), which contains information to supplement instructions to interviewers on the questionnaire itself. Interviewers 
should be instructed to consult their manual if they are uncertain about how to ask a question or how to record 
responses or carry out any other procedure. 


It is especially important that the initial introduction the interviewer gives a respondent is clear and consistent from 
interview to interview. It is common for the text of this introduction to be printed at the start of the questionnaire. 
Usually, interviewers will be instructed to ask questions exactly as they are written in the questionnaire. This is an 
important way to achieve greater reproducibility and standardization between interviewers. 


Whether printed on paper or on an electronic device, the questionnaire should be well designed. If paper is being 
used, the size and quality should be chosen to suit field conditions. Cards are often easier to work with in the field 
than paper sheets but may be unsuitable if more than one is required for an interview and they are also bulky to carry 
around. The layout of the questionnaire should be sufficiently spaced to allow those with large handwriting to record 
all the required information. If whether or not a question is asked depends on the response to a previous question, this 
should be indicated on the questionnaire with clear instructions and appropriate ‘branch and skip’ explanations (see 
Appendix 14.4). If the questionnaire is being administered from an electronic device, it is essential that such branches 
and skips have been correctly pre-programmed (see Section 5.2). 


All questions should be assigned a number. For questions that are repeated several times, such as questions about each 
of a mother's children, a tabular layout can be used (see Appendix 14.5), but this should be designed with care, as 
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such a layout puts more demands on the interviewer, or on the respondent if the questionnaire is self-completed. 


To facilitate later checking and coding, it may be useful to include, on the questionnaire, the names that variables are 
going to be assigned for computer processing (see Section 3.4). These are often typed in capital letters and placed just 
to the right of the coding boxes on the questionnaire. 


3.4. Coding 


Coding is discussed in detail in Chapter 20, Sections 5.4 and 7.3, and only a few points pertinent to questionnaire 
design are covered here. Coding is the process of converting the recorded answers to questions into a numerical or 
alphabetical code. The answers may be numeric (for example, age) or be the replies to closed questions. For closed 
questions, there are two possible ways of coding, depending upon whether only one answer, out of the list of possible 
responses, can be given or whether several are possible. Examples of the former are any ‘yes/no/don’t know’ answers 
or answers to questions such as relationship to the head of household (for example, wife, child, brother, or sister, etc., 
where only one answer is allowed). An example of where several answers on the list are possible for a single 
respondent is a question about food consumed on the previous day. In the first case, the possible responses are each 
given a code, usually a letter or a digit, and a respondent's answer is coded accordingly. In the second, each possible 
response must be coded for the answer ‘no’ or ‘yes’ (often coded as ‘0’ or ‘1’, respectively, or as ‘N’ or * Y?) or ‘don’t 
know’ (if applicable) (often coded as *9"), and the codes for each of them will make up the respondent’s reply. 


It is important to allow codes for ‘don’t know’, rather than leaving the code blank. On paper questionnaires, answers 
to questions that are skipped (i.e. which are not relevant) are normally left blank during the interview. It may be 
convenient to leave the codes blank as well, or a specific code for ‘not applicable’ (for example, *8") can be used. The 
choice depends on data processing requirements (see Chapter 20). With lists of possible responses, a category ‘Other 
(specify) is often included and needs to have its own code. There should be space on the questionnaire to write or 
type in the actual reply, but, as mentioned in Box 14.2, the pre-testing and pilot work should ensure that the *Other 
(specify)’ category is uncommonly used for a reply. 


Appendices 14.1 to 14.10 give some examples of different ways of designing a questionnaire and examples of 
different types of questions. 


4. The interviewers 


4.1. Selection 


Interviewers should be selected with careful attention to the tasks they will be expected to perform. They must be seen 
by the respondents as individuals who can be trusted to keep sensitive and confidential information to themselves. 
They must be of pleasant disposition, and be well-mannered, well-dressed, reliable, and punctual. They must not 
make promises to respondents that they do not honour (for example, if they say they will return on a given day they 
must do so or, at least, send a message in advance to explain and apologize if they cannot). The study investigator 
must attempt to assess whether potential interviewers have these characteristics during initial selection processes, 
which should include written tests and interviews. 


In general, contracts of employment for interviewers should include a probationary period, during which their 
suitability is further assessed and at the end of which a decision about longer-term employment is made. In some 
countries, it is possible to offer initial contracts solely for the training period. If so, the trial can select more 
interviewers that are needed and train them. This has two considerable advantages. It gives the trial team much more 
time for a detailed assessment of their character and performance than is possible through a short written test and 
interview, and also it is possible to select the best potential interviewers at the end of the training period, which should 
have included actual pilot testing of their tasks in the field. Other interviewers who have performed satisfactorily can 
be put on a waiting list, so that they could be offered the job at a later date, without the need for a full training course 
if one or more interviewers drops out or falls sick. Even after this, there must be provision for removing an 
interviewer from fieldwork if their performance is unsatisfactory. 


If possible, interviewers should speak the same language as the respondents; otherwise, interviews will have to be 
conducted through interpreters, which is usually unsatisfactory (for the reasons outlined in Section 2), although it is 
sometimes unavoidable. 
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The sex, age, and normal place of residence of an interviewer may be important. For example, in some societies, male 
interviewers are less likely to get reliable information from women and may even not be allowed by local custom to 
interview them at all. If interviewers clearly belong to the health services, replies may be biased towards support for 
those services. Well-educated interviewers may not be best for interviews with less educated respondents; substantial 
differences in social status between interviewer and respondent should be avoided. Young interviewers may not be 
regarded as reliable or trustworthy recipients of sensitive information by adults but may be best able to elicit sensitive 
information from other young people. Also, sometimes sensitive information may be more readily given to a stranger 
than to a member of the same community, provided the respondents are assured of confidentiality. 


The most skilled and reliable interviewers are not always the most intelligent or highly educated. Indeed, highly 
educated interviewers may be more likely to become bored with repeated administration of the same interview 
schedule, especially if they do not see a clear and feasible career path by which they can progress, for example, to 
being a supervisor or to getting the opportunity for further training. 


4.2. Training 


The training of interviewers might initially be done as a group exercise, with classroom-type teaching. This must be 
supplemented with practical exercises. These might consist of one interviewer administering the survey questionnaire 
to the trainer (or another interviewer or someone else), while others look on, followed by a critical evaluation and 
discussion of the interview with the group. The person acting as the respondent should not be expected to answer any 
sensitive questions honestly but can make up plausible answers. 


The draft interviewers' manual (see Section 4.4) should be used extensively in the training process, so that, by the end 
of training, the interviewer should be familiar with all aspects of the manual and know which parts to consult for 
advice on queries about particular questions or aspects of field procedures. Also, the training process usually reveals 
aspects of the manual that need revision or further clarification. 


Only after interviewers have been through a preliminary training course should they be allowed to try out interviews 
in the community. Initially, such interviews might be done by pairs of interviewers, in the presence of a trainer, with 
detailed *post-mortems' being conducted after each interview or series of interviews. The training process will merge 
with the processes of standardization (see Section 4.3) and validation (see Section 2.4) and should be continually 
reinforced throughout the trial through supervision visits and meetings, and, when necessary, refresher training 
courses. 


4.3. Standardization 


As discussed in Section 2.4, an interview must be both repeatable and reproducible. Standardization of interview 
technique within and between interviewers is necessary for reproducibility. Interviewers must be trained to follow 
instructions on the questionnaire, as well as all other instructions, exactly. This extends to asking all questions exactly 
as written, if this is appropriate. As well as questions, the introduction to the interview, explanations and definitions 
made to the respondent, and transition statements that explain a change of subject of the questions should be said as 
written. The points in the interview to use probes and prompts to get the respondent to reply more fully should also be 
clearly specified. 


Standardization may also apply to the place and time of interviews. For example, interviews conducted in a home and 
health centre will, in many circumstances, produce different responses. 


It is not realistic, however, to insist on interviewers being merely reading and transcription ‘machines’. They must 
have some leeway to add extra explanations and guidance when it is clear that a respondent does not understand a 
question or a definition. Interviewer training should cover this and detail the extent to which this is permissible. 
However, stress should be placed on following the written wording, whenever possible. 


Standardization needs a certain degree of regimentation, and this can act against rapport and personal contact. Since 
the wording of questions is laid down in advance, it is important to ensure that it is friendly and does not alienate the 
respondent. If different interviewers are getting different responses to the same questions, it is important to investigate 
why. For example, one of the interviewers may be deviating from the interview schedule and giving undue emphasis 
to part of a question. Different interviewers attempting to collect the same information from one respondent at 
different times will normally be a part of the validation procedures (see Section 2.4). 
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No matter how well interviewers are trained and standardized against each other, it is as well to assume in the design 
of a survey that some differences will exist in the responses obtained by different interviewers. This will influence the 
way different interviewers are deployed for fieldwork. Not only is it important to record on the interview schedule 
who conducts each interview, so that differences between interviewers can be analysed, but also interviewers should 
be deployed in a ‘balanced’ way, so that interviewer differences are not confounded with other differences of potential 
interest. For example, in an intervention trial, each interviewer should question similar numbers of subjects from the 
intervention and control groups. The worst situation would be for one interviewer to question those in the intervention 
group and another interviewer to question those in the control group. 


4.4. Interviewers' manual 


An interviewers’ manual should be developed for use during the field survey. This should be reviewed during the 
training programme for interviewers and revised, as necessary. Careful version control will be needed to ensure that 
the current versions of the manual and questionnaire match each other. The manual should give detailed instructions 
regarding how individuals are to be selected and approached for inclusion in the study and for each specific interview, 
and it should detail any special instructions regarding each question in the questionnaire and how the responses should 
be entered. It should include guidance on how to deal with unusual situations and how to code unusual responses. It 
should also outline what checks are to be conducted on completed questionnaires and how and when completed 
questionnaires should be submitted for data processing and analysis. 


During the conduct of fieldwork, regular meetings should be held of interviewers to discuss the progress and queries. 
When new problems arise, the solutions should be incorporated into the field manual, so that there is consistency in 
dealing with the problem in the future and a permanent record is kept of the solution adopted. The manuals held by 
each interviewer should be updated regularly, and the text should be kept electronically to facilitate this. Again, 
careful version control is essential, and any changes to the manual (or questionnaire) should be documented in the 
trial diary (see Chapter 16). 


5. Data capture 


Traditionally, data, whether from interviewer-administered or self-completion questionnaires, have been entered 
initially (captured) on to paper, but there is increasing use of electronic data capture. The latter has many advantages 
and has become more generally feasible, as the sizes, prices, and robustness of suitable electronic devices have 
improved. 


5.1. Pen and paper 


The major advantages of data capture by pen and paper are that it involves relatively little capital expenditure and 
does not require interviewers to be familiar with using an electronic device. Also, if an interviewer detects some 
specific problem with a question or an answer code, they can easily make a note of this in the margin of the paper 
form and move on to the next question. However, if many questionnaire forms are being used, the paper becomes 
bulky and heavy to transport and store. Paper forms can easily be damaged by rain, insects, or other animals, and a 
further step of data entry on to computers causes extra expense and delays and can result in transcription errors (see 
Chapter 20). Some projects scan the data from the paper form into a computer or fax the data to a central data 
management facility for subsequent scanning or manual data entry. Such methods require relatively high-quality 
scanners or fax machines and that the questionnaires are completed neatly, using standardized writing styles to avoid 
transcription errors. 


5.2. Electronic 


Electronic data capture involves either the interviewer or the respondent entering the responses directly into an 
electronic device. This allows electronic range and consistency checks to be done at the time of data capture when it is 
still possible for the interviewer or the respondent themselves to correct a mistake or misunderstanding that leads to 
an ‘impossible’ response. Even in most rural areas of low-income countries, electronic devices, and especially mobile 
phones, are now widespread, so using them for data capture is now rarely likely to faze respondents, though this must 
be checked in preliminary pre-testing and pilot studies. 


Some electronic devices incorporate GPS (see Chapter 10), so that the coordinates of a household or other interview 
location can be recorded, and the device can even be used to guide the interviewer to the same location subsequently. 
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Many electronic devices also incorporate an audio function, so that the respondent can listen to the questions and 
answer options through a loudspeaker or earphones. It is also possible to allow the respondent to have the question 
repeated, and, if necessary, they can be allowed to go back to correct an earlier answer. 


Data captured onto one electronic device can be easily transferred to another. It is essential that all such data are kept 
confidential through password protection and, when applicable, encryption. 


Until recently, initial hardware, software, and programming costs prohibited the widespread use of electronic data 
capture in field trials in LMICs, but there are now cheap smartphones that are capable of displaying a substantial 
questionnaire and capturing data in a way that is very simple for interviewers or respondents to use. There is also free 
user-friendly software that can be programmed by non-specialists for questionnaire design and data capture (for 
example, <http://opendatakit.org>). 


Although electronic data capture has major advantages over traditional pen and paper approaches, it is important to 
allow sufficient time for someone on the team to fully familiarize themselves with the hardware and software to be 
used and for electronic questionnaire development and careful testing. All programming *bugs' need to be ironed out 
before fieldworker training starts, and procedures need to be carefully tested and rechecked during training and in any 
pilot test. Unlike with a paper questionnaire, a problem with the programming of an electronic questionnaire can 
result in it being impossible for an answer to be entered or for the respondent to move on to the next question, being 
made to skip questions they should have answered, and even for whole batches of data being lost, for example, during 
data transfer. Furthermore, during the early stages of transferring from using paper questionnaires to electronic 
devices, all investigators, data managers, and fieldworkers must become fully familiar with the new method and 
device, and someone must be immediately available to solve any unexpected problems that arise (see Chapter 20, 
Section 5 for further details). During the transition period from a team using pen and paper to electronic data capture, 
it is often a sensible precaution to give the interviewers paper versions of the questionnaire as a backup, in case there 
is some unexpected problem which makes the electronic version unusable, at least during the pilot test and perhaps 
the first few days of the main survey. This is particularly important if the interviewers will be a long way from the 
trial's coordinating centre. 


Some examples of using mobile phones to capture different kinds of data are given in Appendix 14.8, Appendix 14.9 
and Appendix 14.10 


6. The interview 


As much as possible, a face-to-face interview should approximate to a conversation between the interviewer and 
respondent and must not be an interrogation. Good rapport between the two is vital, and the onus is entirely on the 
project team to ensure this. 


6.1. Who, where, and when 


In studies of children, the best informant regarding their health or behaviour is likely to be their mother or guardian. 
Only as a last resort should someone else be interviewed for this purpose. This may necessitate repeated visits to a 
household, until the mother or guardian is at home. Other than for children, proxy informants should be avoided, if 
possible. 


The choice of the place of interview will be influenced by logistic considerations and the nature of the information to 
be collected. Usually, the place will have to be chosen for the convenience of the respondent, rather than for that of 
the interviewer. Privacy will be easier to ensure in a hospital or a clinic than in a village setting, but special 
arrangements may be made to ensure greater privacy in a village. For example, an interview might be conducted 
slightly away from the house under a shady tree. If interviews are to be conducted in homes, as far as possible, the 
time should be chosen to fit the convenience of the residents. If possible, they should be consulted, or at least 
informed, in advance regarding when an interview will be scheduled. Preliminary investigations, before the main 
survey, should be made to ascertain when the most convenient time will be for most participants. In rural 
communities, during planting or harvesting seasons, evening interviews may be preferred. But, if interviews take 
place after dark, poor lighting may be a problem, and attention to clear printing and a well-spaced layout for the 
questionnaire becomes even more important (as well as the provision of torches and batteries to interviewers). In 
some areas, security after dark may also be a significant problem, and interviewing at that time may be inadvisable. 


6.2. Non-response 
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Steps that can be taken to ensure data completeness within a single questionnaire are discussed in Chapters 16 and 20. 
Here, we discuss the problem of non-response where a trial participant is either not seen or refuses to take part in the 
trial or in a particular data collection ‘round’ or survey. 


Non-respondents in a study are rarely representative of the rest of the study population. They are a self-selected 
group, and thus their exclusion will usually introduce bias into the results of a survey, but the degree to which that has 
occurred is not usually directly measurable. Thus, if a high proportion of the target population for interviews are not 
interviewed, the valid interpretation of the results from those who are interviewed, and in particular the generalization 
of these results to the whole community, may be open to serious question. Therefore, great care must be taken to 
ensure that the response rate is high. This may be achieved in several ways. First, the questions included in the 
questionnaire should be thoroughly tested in a pilot study, so that any that a significant proportion of respondents 
cannot, or will not, answer adequately are eliminated. Second, an appropriate explanation of the survey should be 
given to study participants in advance, and any false suspicions they have about the motives or intentions of the 
investigators must be dispelled. Third, interviewers must be selected who are persistent, yet polite, and who will probe 
for a correct response to a question and not accept a *don't know' response too readily. Fourth, interviewers must be 
instructed to call back repeatedly if a house is empty or a respondent is away, before abandoning an interview. Their 
work schedule should take into account the need for such return visits. 


Systems should be put in place to monitor the non-response rate within a trial on an ongoing basis, so that steps can be 
taken to attempt to decrease this, before it is too late. The non-response data should be disaggregated by the 
interviewer, the trial team, and other important groupings, where appropriate, such as language, location, etc., and all 
outliers investigated carefully. For example, in a trial of human papillomavirus vaccination within schools, it was 
discovered that the non-response rate was substantially higher for one field team than for the other. This turned out to 
be due to the way the team members were introducing themselves and the trial within the schools—something that 
could be changed, and the problem was quickly solved. 


Even in the most well-conducted surveys, a 100% response rate is rare. Indeed a 100% response rate should be 
viewed with some suspicion! As much information as possible should be obtained about non-respondents, where 
necessary from proxy informants, so that the characteristics of non-responders for which information is available (for 
example, age and sex) may be compared with that available on responders. This may give clues to the extent of 
possible biases resulting from their exclusion. 
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Appendix 14.1. Options for recording responses on a questionnaire 


There are many ways in which the responses to a question can be recorded within a questionnaire. Three of the 
commonest ways are shown in Figure A14.1. 


Appendix 14.2. Pre-coded responses which are mutually exclusive 


If there are multiple potential responses to a question, but these are mutually exclusive, so only one answer is 
permitted, then it is possible to use a layout as in Figure A14.2. 


Appendix 14.3. Pre-coded responses which are not mutually exclusive 


If there are multiple potential answers to a question, but these are not mutually exclusive so multiple responses are 
permitted, then each option must have its own response (for example, Yes/No) within the questionnaire (Figure 
A14.3.). 


Appendix 14.4. Questions with a 'skip' instruction 
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Some questions on a questionnaire may not be applicable for some respondents, based on their answers to earlier 
questions. Although it is possible to design the questionnaire so that a special ‘Not applicable’ code is allocated for 
any such questions, an alternative is to design the questionnaire to allow the respondent to skip such questions. An 
example is given in Figure A14.4. 


Appendix 14.5. Recording of multiple items of information for direct computer entry 
Sometimes, one needs to ask a series of questions about each person in a group (for example, household). In such 
situations, the questions series might be structured in a table, as shown in Figure A14.5. 

Appendix 14.6. 'Open' questions 


Although open questions should be avoided if possible in questionnaires, as collating and post-coding such questions 
can be very time consuming in large studies, occasionally it is essential to have an open question. An example is given 
in Figure A14.6. 


Appendix 14.7. Questions for self-completion by the respondent 

Questionnaires can be designed for the respondent to enter their responses directly, rather than this being done by an 
interviewer. Figure A14.7. gives two examples. 

Appendix 14.8. Questionnaires on a mobile phone 


Increasingly, questionnaires are being designed so that the responses are recorded directly into an electronic device 
such as a computer (desktop, laptop, tablet), PDA, or mobile phone. This can be done either by an interviewer or the 
respondent themselves. An example is given in Figure A14.8., which shows photographs of mobile phone screens 
showing data on sexual behaviour that has been entered directly into an Android phone using Open Data Kit (ODK) 
software by three different trial participants. 


Appendix 14.9. Collecting geolocation data on a mobile phone 


Electronic devices can be used to collect geolocation coordinates if the device has that facility. Figure A14.9. 
reproduces photographs of mobile phone screens showing data on the coordinates (geolocation) where a questionnaire 
has been completed that have been entered directly into an Android phone, using ODK software. 


Appendix 14.10. Recording a laboratory test result on a mobile phone 


Electronic devices can be used to collect photographs if the device has that facility. Figure A14.10. reproduces 
photographs of mobile phone screens showing a photograph of a pregnancy test result that has been taken using an 
Android phone which will be saved for the record. 


https://www.ncbi.nlm.nih.gov/books/NBK305523/"?report-printable 11/24 


22-4-2020 Questionnaires - Field Trials of Health Interventions - NCBI Bookshelf 


Figures 


Figure A14.1 


Three alternative options for recording the response to a single question (face-to-face interview using pen and 
paper). 
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CODE FOR 
COMPUTER 


Q49  Whatis the main source of drinking water for members 
of your household? 


(CIRCLE ONE RESPONSE) n 
Piped into residence 
Piped into yard or plot......................... ss sssssssssss 02 
Public Dij. oco pry Senet EMEP EER RT eee eat 03 
Well with hand-pump 04 
Well without hand-pump.................... sees seres 05 
River, spring, surface water 06 
Tanker truck or other vendor 
ENEMIES rcd at c IOVIS AN 
Other (specify) 
Not ROW seess rin imponi n GN IS ASADOFOEIX UL vili d 
Note: A special code has been allocated for ‘Not known —in this case, 99. It is 
good practice to use a standard code for answers such as ‘Not applicable (such as 
serial 8s (8 or 88 or 888...)) or ‘Not known (such as serial 9s (9 or 99 or 999...)). 
In this example, it would also be possible to subdivide and add further codes for 
the ‘Other’ responses at the analysis stage. This would require that all the ques- 
tionnaires coded ‘09’ were re-examined, and the responses given new codes such 
as 10, 11, 12, and so on. Such re-coding is time-consuming, and it is usually bet- 


ter to try to ensure during the pilot study that all, or almost all, the responses will 
fall into the specific coded categories to avoid having to do such later re-coding. 


Figure A14.2 


Design of a question with multiple, mutually exclusive responses (face-to-face interview using pen and paper). 
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CODE FOR COMPUTER 
Q33 Which of the following items does your household own? 
(ENTER NO = 0 OR YES = 1 FOR EACH ITEM) 


Sewing machine 


Bicycle 


MOUIIOE i ces y OECD ER 


Figure A14.3 


Design of a question with multiple responses that are not mutually exclusive (face-to-face interview using pen and 
paper). 
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Figure A14.4 


Design of a sequence of questions which allow questions that are not applicable to be skipped by appropriate 
respondents (face-to-face interview using pen and paper). 
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Q93 For all your children who were born alive, give the following details 


(starting with the first born): 


Child Name 
number 


Alive(1)or Sex 
(1 = Male; 2 = Female) 


dead (2) 


01 
02 
03 
04 
05 
06 
07 


LLLILLLILILILE 


IF NUMBER OF CHILDREN IS DIFFERENT FROM TOTAL NUMBER RECORDED IN Q40, 


PROBE AND RECONCILE. 
* IF DEAD, GIVE AGE AT DEATH. 
RECORD AGE IN COMPLETED YEARS AND MONTHS. 


Age* 


Years 


D 


x 
=a 


IF LESS THAN 1 MONTH, RECORD AS '00' YEARS AND ‘00° MONTHS. 


Figure A14.5 


Design of a form to record a series of questions about children born to the same woman (face-to-face interview 


using pen and paper). 
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Q82 What do you do when your child has diarrhoea? 
(RECORD MOTHER'S RESPONSE) 
(PROMPT: DO YOU INCREASE OR DECREASE FLUID INTAKE?) 
(PROMPT: IF YOU SEEK HELP, WHO DO YOU GO TO FIRST?) 


Note: The prompts are included, so that the interviewer is asked to enquire about 
these issues if the mother does not volunteer the information spontaneously. 

If the responses to this ‘open’ question are to be analysed quantitatively, the 
information must be coded after all the questionnaires have been completed and 
the full range of different responses has been assessed. If the survey is large, this 
can be a lot of work, and it is usually better to have explored this in pilot studies, 
so that as many questions as possible are in the form of 'closed' questions on all 
questionnaires that will be analysed quantitatively within the main trial. 


Figure A14.6 


Example of an ‘open’ question (face-to-face interview using pen and paper). 
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Figure A14.7 


Example of questions for self-completion by the respondent (pen and paper). 
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Participant 1 


Participant 2 


OOK Collect questionnaire ODK Culleco questionnaire OOK Collects questionnaire OOK Collect questionnaive 


Have you ever had sex? Have you ever used How many sexual How many sexual 


acondom? partners have you ever partners have you had in 


had? the last 12 months? 
O Yes 


[9] No 


Q 1 don't want to 
answer 


Lf Caen "prn at densior Era Vp rz MN E s 

| Based on the "No" response to the "Have you ever used a I 
| condom?” question, the form was pre-programmed to skip the | 
question "Did you use a condom the last time you had sex?" I 


| 
L 
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ODK Collect» questionnaire ODK Collect» 


Have you ever had sex? Now I'm going to ask you 
some questions about 
O Yes your medical history. 


@ No 


» 
| Based on the “No” response to the “Have you ever had sex?” | 
| question, the form was pre-programmed to skip to the end of the | 
| series of questions about condom use and sexual partners I 
ee nm acuan cf eS p Da 

Figure A14.8 


Examples of questions answered directly on a mobile phone by three different study participants. 


Reproduced courtesy of Zachary Kaufman and Rebecca Hershow, GOAL Trial, South Africa. This image is 
distributed under the terms of the Creative Commons Attribution Non Commercial 4.0 International licence (CC- 
BY-NC), a copy of which is available at http://creativecommons.org/licenses/by-nc/4.0/. 
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ODK Collect» questionnaire 


Press the button below 
to show the location 
where the interview was 
conducted 


e UN photo 


The location is shown 
below. Swipe the screen 
to continue. 


Latitude: S 33955'18" 


[i] Loading Location . Ju 

" 2 Longitude: E 1892516" 

Record Location =p Altitude: 178.7m 
Using Accuracy is 2] r 


Accuracy: 23m 


— 


E OU ee ee ee ae ee SP NO I REV E PES Ww cm a P. e Fh AE ERE Ey dé TAN aa 2 
| Geolocation can be captured on suitable devices | 
Ms ‚Á IÍÉIÍ|IÍÉ|IÍ|IlÍÉIllIlIIIIIIILILILILL—L—————————————————— E 
Figure A14.9 


An example of collecting geolocation data on a mobile phone. 


Reproduced courtesy of Zachary Kaufman and Rebecca Hershow, GOAL Trial, South Africa. This image is 
distributed under the terms of the Creative Commons Attribution Non Commercial 4.0 International licence (CC- 
BY-NC), a copy of which is available at http://creativecommons.org/licenses/by-nc/4.0/. 
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Figure A14.10 
Example of a photograph taken with a mobile phone as part of a questionnaire. 


Reproduced courtesy of Zachary Kaufman and Rebecca Hershow, GOAL Trial, South Africa. This image is 
distributed under the terms of the Creative Commons Attribution Non Commercial 4.0 International licence (CC- 
BY-NC), a copy of which is available at http://creativecommons.org/licenses/by-nc/4.0/. 
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Boxes 


Box 14.1 Checklist for the development of a questionnaire 


l. 


Define the information that is required from the questions. Some items of information may only require a 
single question, such as name or sex, while others require a series of questions such as socio-economic status 
or episodes of ilIness in the past week. 


Formulate draft questions. Attention to the wording of questions is important, as slight variations may result 
in different responses. For example, ‘Where do you normally seek help when your child has diarrhoea?’ vs 
*Where did you seek help when your child last had diarrhoea?'. Box 14.2 gives a checklist of points that 
should be considered in drafting questions. In general, it is a good idea to search for, and to critically review, 
how others have asked specific questions, especially if these questions have been formally validated. 


Informally test the questions. This may involve trying them out on different members of the study team and 
discussing them with those knowledgeable of the study area, including residents. It may be necessary to base 
someone in the community under study (ideally, someone with anthropological or social science skills) to 
investigate how different questions will be perceived to find out if there are taboos regarding certain topics, if 
there are local words for some illnesses or conditions, and the extent to which these correspond to the 
investigator's definitions (for example, many communities have special words for measles, night blindness, 
sexual intercourse, depression, or lethargy). The investment warranted for such qualitative studies will 
depend upon local sensitivities regarding the items on which information is required and the degree to which 
each question is critical for the trial. For example, it will require less work to find out how to ask questions 
about breastfeeding practices than to formulate appropriate questions on aspects of sexual behaviour. As a 
result of such investigations, the original draft questions may have to be modified. Some may even have to 
be abandoned if research indicates that valid information is unlikely to be elicited through a questionnaire 
survey. 


Prepare a first draft of the questionnaire for pilot testing. 


Translate each question into the language(s) of the study population, followed by independent back- 
translation by someone who does not know the original questions, with reconciliation of any discrepancies— 
ideally followed by further independent translation and back-translation (see Section 2.5). 


Prepare a draft instruction manual for interviewers and their supervisors (see Section 4.4). 


Pilot-test the questionnaire in field conditions, preferably in an area adjacent to the study area and using the 
interviewers who will work on the main survey (see Chapter 13). 


Analyse the experience in the pilot test and the data collected. 


Reformulate the questionnaire, with further translation and back-translation of any amended questions, 
followed by further pilot testing, especially if important changes have been made to questions related to 
primary or secondary trial outcomes. 


10. Finalize the questionnaire for the main survey, along with the instruction manual for interviewers (see 


Section 4.4). 


Box 14.2 Checklist of points to consider when drafting questions 


l. 
2. 


3i 


Keep wording informal, conversational, and simple. Avoid words longer than three or four syllables. 


Avoid jargon and sophisticated language; assessing understanding at the pre-test and pilot test stages is 
essential. The wording of all questions must be appropriate to the educational, social, and cultural 
background of the respondents. 


Check the cultural relevance to the respondents of concepts used. Ensure mutual understanding between the 
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interviewers and the respondents, paying attention to cultural and educational differences. 


4. Avoid long questions, but vary the length of questions to avoid administration of the questionnaire becoming 
repetitive and boring for the interviewer or interviewee. 


5. It may be necessary to define a term or a concept before asking about it. If the definition is short, it can be 
included in the question, but otherwise it is better given separately before the question is asked. 


6. Avoid leading questions that may bias the respondent to a particular answer (for example, ‘Do you think the 
improved clinic arrangements are better?"). 


7. Avoid open questions beginning ‘Why?’. 


8. Avoid negative questions (for example, ‘Do you not think . . . "—in some cultures, the answer ‘no’ indicates 
‘I do not think . . . ^; in other cultures, the answer ‘yes’ indicates ‘Yes, I do not think . . . ’!). 


9. Where possible, avoid hypothetical questions, as some respondents will find these difficult to answer (for 
example, ‘If the bus fare was less, would you come to the clinic more often?"). 


10. Keep to a single subject for each question. For example, do not say ‘Do the cost and times of the clinic 
prevent you going?’. 


11. Pay particular attention to sensitive issues. Review the inclusion of very sensitive ones. If they are to be 
retained, pay very careful attention to the wording, and consider the use of indirect approaches. Think 
carefully about their position within the questionnaire (see Section 3.2). 


12. Check the adequacy of the lists of responses to ‘closed’ (see Section 2.3.2) questions. For example, ensure a 
food list covers most things normally eaten in the community concerned. It is usually a good idea to include 
an ‘other (specify)’ category, unless you are sure that every possible answer is in the list (such as male and 
female for gender). But it is also important that only a relatively small proportion of responses (definitely 
less than 10%) end up being in the ‘other (specify) category. This should be checked in the pilot test, with 
additional categories being added for the commoner responses that were initially in the *other (specify) 
category. 


13. Never include an ‘other’ category without asking the respondent to specify what the response was—as in 
‘other (specify)’—and leave space for the respondent or interviewer to write the specific answer next to this 
code. 


( London School of Hygiene and Tropical Medicine 2015. 
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